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SIGNIFICANCE  AND  EXPLANATION 

Linear  prograonlng  deals  with  the  problem  of  mlnlmizinq 

T 

c X 

subject  to 

Ax  ^ b,  X ^ 0 . 

This  has  been  spectacularly  successful  in  certain  classes  of  optisiization 
problems  that  occur,  for  example,  in  management.  However  most  situations  are 
inherently  nonlinear,  and  one  would  like  to  be  able  to  deal  with  the  general 
problem:  minimize  f (x) , subject  to  g{x)  ^0,  x ^ 0.  Progress  in  develop- 
ing  practical  and  efficient  methods  for  this  full  nonlinear  problem  is  slow, 
as  one  might  expect  due  to  its  generality.  The  present  paper  deals  with  an 
efficient  computational  method  for  solving  the  restricted  problem:  minimize 
f (x) , subject  to  Ax  ^ b,  X ^ 0,  i.e.  minimize  a nonlinear  objective  func- 
tion with  linear  inequality  constraints. 

The  method  depends  on  starting  with  a feasible  solution  x^,  i.e.,  a 

solution  that  satisfies  the  Inequality  constraints.  Then  produce  a sequence 
x^,X2,x^,...  of  feasible  solutions  by  a variable  metric  method  (see  2d}Stract) . 
Under  suitable  assumptions  these  will  converge  superlinearly  to  the  optiaul 
solution. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 


A VARIABLE  METRIC  METHOD  FOR  LINEARLY  CONSTRAINED  MINIMIZATION  PKvBLEMS 

KIau*  Ritter 


1.  Introduction 

For  neny  years  variable  aetric  methods  have  been  used  successfully  in  unconstrained 
minimization.  In  19b9  Goldfarb  |7]  extended  the  Oavidon-Fletcher-Powell  SMthod  |9| 
to  problems  with  linear  equality  and  inequality  constraints.  In  (6)  Gill  and  Murray 
described  variable  aetric  methods  for  linearly  constrained  prcblesis  which  use  approxiaa* 
tlons  to  the  Hessian  matrix  of  the  objective  function  rather  than  to  the  inverse 
Hessian  matrix.  In  all  cases  it  has  only  been  shown  that  the  method  determines  the 
optisial  solution  in  a finite  number  of  steps  if  the  objective  function  is  convex  and 
quadratic.  Recently.  Fischer  14)  proved  superlinear  convergence  of  the  Davidon-Fletcher- 
Powell  and  the  BroydeirFletchor-Goldfarb-Shanno  (11]  method  for  linearly  constrained 
problems. 

Brodlie,  Ckjurlay  and  Greenstadt  (2)  and  more  recently  Davidon  [3]  have  investigated 
variable  metric  methods  where  the  matrix  which  approximates  the  inverse  Hessian  of  the 
objective  function  is  factorized.  In  this  paper  such  a factorized  variable  metric 
method  for  lineprly  constrained  problesis  is  given.  Using  Fischer's  results  (4)  it  is 
shown  that  it  converges  superlinear ly.  In  the  unconstrained  case  it  reduces  to  a method 
which  was  investigated  in  (10)  and  shown  to  be  a factorized  version  of  the  Broyden- 
Fletcher-Goldfarb-Shanno  method. 


Sponsored  by  the  United  States  Army  under  Contract  No.  l>AAa29-7S-C-0024 . 


a.  G«n«r«l  d>»criptton  of  th«  «lqorlthw 


We  consider  the  following  alnlsilzstlon  probleaii  Nlnlsilxe  the  function 

nic) 

subject  to  the  constraints 

A*  < b , 

where  x ( e".  b ( e"  and  A Is  an  (•,n)-autrix. 

niroughout  the  paper  it  is  assusied  that  the  set 

R ■ (x|ax  ^ b) 

of  feasible  solutions  In  nonempty  and  that  for  every  x « R the  gradients  of  the 
constraints,  active  at  x.  are  linearly  Independent.  Furtheraore.  we  assisee  that  F(x) 
is  twice  continuously  differentiable  and  denote  the  gradient  and  the  Hessian  aatrlx  of 
P(x)  at  a point  x^  by  g^  • 7F(Xj)  and  ■ G(x^),  renpectivaly . In  order  to 
prove  superlinear  convergence  we  finally  need  the  following 
Assumption  1 i There  are  positive  numbers  w,  n and  L such  that 

(2.1)  m||x|I^  < x‘G(y)x  < ijlxll*  for  all  x.y  i e" 

and 

||g(x)  - G(y)||  lIIx  - y|l  for  all  x,y  « e"  . 

Assumption  (2.1)  Implies  that  F(x)  is  uniformly  convex  and  that  there  Is  a unique 
s ( R such  tliat 


r(s)  ^ P(x)  for  all  X r R . 

let  X . ( R be  a point  determined  by  the  algorithm.  For  ease  of  notation  we 


assusM  that 


•ixj  - Wj.  i 1 


ajIXj  < (b)^,  1 

where  a*,..., a*  denote  the  rows  of  A.  Set 
X M 


q ♦ 1, 


Tj  - {x|a'x  - 0.  i • 1 q) 


and  denote  the  ortlxogonal  projection  of  g^  onto  by  g^.  Then  g^  can  be  written  as 

’3  ■ jx  * ’J  • 


(2.2) 


Because  F(x)  is  convex  it  follows  frosi  the  Kuhn~Tucker>condittons  (sea  e.g.  (8|) 


that  x^  is  an  optiSMl  solution  if  and  only  if 

" 0 and  1^  i 0,  i • 1, 


Suppose  x^  is  not  an  optuMl  solution.  Then  we  want  to  deteraine  a search  direc- 
tion s^  and  a step  sise  > 0 such  that 


(2.3) 


*1*1  ■ “j  ■ “j'j  * " •'‘Vl’  " '■‘“l*  • 


In  order  to  guarantee  that  there  exists  an  with  the  properties  (2.3)  we 

need  a search  direction  s^  with 
(2.4) 


qjsj  > 0 and  aJSj  >0.  i - 1 . 


If  q^  ^ 0 we  can  find  an  s^  * with  qjs^  > 0.  In  this  case  a's^  • 0,  I • l....,q, 

and  all  constraints  which  are  active  at  x.  are  also  active  at  x,  ..  lf>  say.  1 >0 

3 j*l  <l 

we  can  determine  an  s^  such  t(iat  q*s^  > 0,  a^s^  > 0 and  a^s^  •0,  i • 1 q * 1. 

In  this  case  the  constraint  a'x  < (b)  is  net  active  at  x.  .. 

q - q j*l 

It  is  well-)(nown  t)uit  in  order  to  prevent  xiq-zaqqinq  the  decision  to  drop  an 
active  constraint  has  to  be  made  with  some  caution.  Often  it  is  based  on  a comparison 
between  ||q^||  and  the  swiximal  value  of  the  multipliers  defined  by  (2.2). 

We  shall  iKlopt  the  policy  to  choose  s^  r unless 


(2.5) 


Iq^ll  < max(X^ . 


Here  is  a convergent  sequence  of  positive  nuadiers  with  the  property  that 

lim  Y - 0 
j-K)  ^ 


if  and  only  if  s^  f for  infinitely  many  j. 

In  order  to  compute  q^,  X^>...«X^  and  an  s^  with  the  properties  (2.4)  we 
associate  with  each  x^.  detersiinsd  by  the  algorithm,  a nonsingular  (n.n)  -matrix 

■ ‘=13 ‘^nj’  • 


The  colusins  c , ......c  , are  chosen  in  such  a way  that  they  form  a basis  of  the 

q+1.3  nj 

(n-q) -diSMnsional  subspace  T ^ . For  i • l.....q  the  vector  is  than  uniquely 
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drtenilned  by  the  equations 


“k'ij  ■ ‘ “ * ‘ 

*i'lj  ■ ‘ 

1 » • 


Because  c , ^ fora  a baaia  of  T.  it  followa  that  tha  aatrix 

q>l*j  nj  J 

"i  ■ iXi 

is  positive  definite  on  the  subspace  Ty  furtharBore. 


H^x  - 0 for  every  x * span(a^< . . . .a^) 


Multiplyinq  (2.2)  by  wo  )Miva 


Thus 


and  (2.S)  becoaes 


(2.6) 


’J  " ’j  • X 

»j  - ? 1 - 1 q) 


If  (2.6)  is  not  satisfied  we  choose 


otherwise  we  set 


•j  " "j’j 


1 ql 


where  i ^ xsy. 


Mith  s^  deterained  wo  can  define  the  ■axiital  step  size  as  follows 


• ■ '*”1  I 


for  all  i with  a 


:-i ' “]  ■ 


where  we  set  > > if  a^s^  ^ 0 for  all  1.  Pollowinq  a SMtliod  suqqestad  by  Powell  (9| 


we  coapute  a such  that 


1 


I ' 


»V' 


■ V,Vj 

and 

Wj 

with  • 1 if  poaaibla.  Here  and  6^  are  constanta  with  0 ' 6^  < < 1 and 

Sj  < 0.5.  Finally  we  aet 

• iiinCo^.  and  " “j  ” ‘ 

In  order  to  coaplete  a cycle  of  the  alqorltha  we  have  to  coa^JUte  C.^,*(c,  . .....c  i. 

)♦!  l.J*l  n,)»i 

Depending  on  the  conatrainta  active  at  arxl  *j*i'  reapectively * there  are  four 

different  .-ases  to  be  conaidered. 

Case  1;  s^  • and  < Oy  i.e.,  the  aaaie  conatrainta  are  active  at  x^  and  *j*|- 

Therefore,  + j chooae 


^.jel  ■ 'ij'  ‘ ■ ‘ • 

In  order  to  obtain  auperlinear  convergence  we  deteraiine  a new  baaia  c , ..,,...>c 

q*l,J»l  n,J«l 

for  Tj  auch  that 


“i*'  ■ i4m 


satisfiea  the  quaei-Newton  equation,  i.e.. 


where 


To  this  end  coaipute 


- ’in 


and  Pj  - ||Sjl 


il  sj(g^  - gj^j) 


‘'ij’j+1 


. , (i  ■ n II  . !inlt 


and  set 
(2.7) 


"1,1*1  ■ "il  * “il*l'  “ ’ * ‘ " • 


It  can  be  shown  (see  (10))  t)iat  then 


H,.,  - H.  ♦ -1-i P,P 


•1*1  "j 


(d'Pi) 


ri 


I 


In  th«  unconstrained  case  this  is  the  Broyden-rietcher-Goldfarb-Shanno  update  forsula 

for  (see  e.q.  Ill|).  Clearly,  ■ p^. 

It  reaains  to  be  shown  that  the  vectors  defined  by  (2.7)  define  a basis  for  T^, 

Since  s.  < T.  we  have  c , , ( T.  for  i ■ q * l,...,n  and  it  suffices  to  show  that 
3 3 i.l^l  3 

the  i • q ♦ l,...,n,  are  linearly  independent  or  equivalently  that  la 

positive  definite  on  T^.  Let  d^  and  q^  denote  the  orthoqonal  projection  of  d^ 
and  q^  onto  T^,  respectively,  and  choose  u r spen(d^,q^)  such  that  p*u  • 0 and 
u d 0 it  d^  and  q^  are  linearly  independent.  Since  ® every  s « 

can  be  written  in  the  fore 


X • y ♦ pdj  ♦ lu  , 

. • * * 
where  (H^d^) 'y  • (H^q^Iy  • 0.  Observinq  tivat  H^d^  ■ H^d^  and  H^q^  ■ ***  have 


H,y  ♦ ♦ 3*u'H,u>0 


3 


If  X d 0 . 


Case  2:  s^  ■ c . and  o.  < a.,  i.e.,  thi  constraint  a'x  < (b)  is  not  active  at 

3 <l3  3 3 q - <1 


“j+1- 


Since  c ^,...,c  ^ form  a basis  of  T. . we  can  set 
qj  nj  j^q 


With 


Vi  ■ ^ ' ■ • 

sis  of  we  can  set 

3^q 

l.J+1  ■ ®ij'  i - ♦ 1 n . 


Vi  ■ ‘'l.3+l®i,j*l 


we  have 


H^^^x  ■ HjX  for  X * Tj 

because  c ^ is  orthoqonal  to  c .,...,c_..  If  we  set 

qj  hj 

■ '*>  ■ vS ‘ • ‘ ■ ‘ 

then  these  vectors  are  orthoqonal  to  and 


‘ll‘'i.3+l  ’ - i'  i “i'=k,J+i  • 


1 . 
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Caae  3:  s ■ H.q.  and  0.  • o.,  a new  constraint  la  acttve  at  x List  a bs 

3 3 3 3 3 3**  <*♦* 

the  qradient  of  this  new  active  constraint.  Then 


T -(x|a;x.O.  i-l <,  ♦ U 


Since  we  consider  as  an  approx lauit ion  to  the  inverse  Hessian  aatrix  of  r<x)  on 

the  subspace  T.  we  want  to  deteraiine  a basis  c of  T.^,  such  that 

3 <j4-2.3*X  n,3*l  3*1 


’ 3^*2 


"jtl*  ’ “j*  “ ' ’■j*!  • 

This  can  be  done  by  usinq  a Icna  qiven  in  (1|  which  in  our  notation  is  as  follows. 

Leasu  1:  I.et  v “ H,a  and  w • v'a  ..  - c' ,a  where  v r {q  ♦ l,...,n).  If 
j q+1  q*l  vj  q*l  ^ 


w • 0 set 


'■i.j+l  "ij" 

otherwise  set,  for  i • q ->  l,...,n,  i ^ v. 


.3*1  • '13*  ' ■ ^ ' ‘ ' 


c -c  - c'  a ^ V tc'  a 

i,j+l  ij  i3  q+1  ''**q*l 


-'i*  * Ic'.a 

vi  q*l  V v3  q*l 


is  a solution  of  the  equation 


ut  -fJc'a  t-l-O. 
v3  q*l 


-q*l‘=i.3*l  ■ 


spanfv.c^^,  i » q ♦ l,...,n,  i ^ v)  • span(c^^j  ^ , . . . ) 
n.3*lV3*l  ■ n,  i^v.  h^v. 


n.3*lV3*l  ■ ^ n,  i^v.  h^v. 

'’kj''‘k.3*l  * ^1*1  "j^.3*l  ■ ''k.j*!  "3^3  ■ ^3- 


-7- 


I 


Cl 


early,  c ,,  i«q«l,...,n.  fur*  a baale  for  Furtheraorc 

i » !♦ I 


l^v 


which  iaplles  * * ^1*1” 

10  define  the  renalninq  coluans  of  let 


%♦!  ’ *q*i 


j.  “iiV.’*.  • 


Then  a Is  the  orthoqunal  projection  of  a , onto  T and  therefore  orthoqonal  to 
q+1  q*i  J 

a,...., a . Since  a , is  also  orthoqonal  to  T,.,  each  of  the  vectors 
1 q q*l  )♦! 


v,J+l 


■ 

*q*l*q»l 


i.jti  ■ 'ij  • '•qn^i’S.jti'  ^ ^ ' 


is  orthoqonal  to  property  that 

- 0,  k <*  i,  k « 1,... 


• 1 . 


Case  4t  s • c . and  a.  ■ o.,  i.c..  Instead  of  the  constraint  a'x  < (b)_  a new 

qj  j j q - q 

constraint,  say,  aV.x  < (b)  , is  active  at  x..,.  If  all  q ■ n,  i.e.,  if  x. 

q-fl  — q+1  )+l  J 

is  an  extresie  point  of  R,  set 

!aL 


'q*l%l 


-Si. 


c.,  l"l,...,n,  i<4q. 


-4+l'qj 

If  q < n use  the  procedure  of  Case  3 to  add  the  constraint 

a'  ,x  < (b)  , to  the  set  of  active  constraints.  Denote  the  resultinq  siatrix  by  C.^,. 

q+1  — q+1  j+l 

Then  use  the  siethod  of  Case  2 with  C^  replaced  by  ^ drop  the  constraint 

a'x  < (b)  . The  resultinq  astrlx  H..,  has  the  property 
q - q Jtl 

"j*l"  “ "j*  - 0,  1 - l,...,q  ♦ 1}  . 


r®/'  - 

fei: 
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3.  Detailed  sf  te»ent  of  th«  algorithm 

It  Is  sssuned  that  the  al9orithB  starts  with  an  extrasie  point  of  the  feasible 

region  R which  can  be  obtained  by  solving  a linear  einisiization  problesi.  Let 

*1*0  “ ****  i'  1 " 1. . . . .n 
*1*0  * i " n ♦ 1, . . ..SI  . 


O'  - (a a ) and  C„  • D 

in  u 


‘'lO ‘"nO*  ' 


then  the  matrix  has  the  properties  described  in  the  previous  section.  In  addition 

to  the  matrix  we  associate  with  each  x^.  generated  by  the  algorithm,  a set 

'’<*!»  • ‘“ij “nj’ 


where  r (0,1, . 

.,,m).  If 

“ij  ■ 

then  c^^  is  orthogonal  to  the  gradients 

of 

all  constraints  active  at  Xy 

If 

■ k > 0,  then  the  constraint  a^x  ^ (b)j^ 

is 

active  at  x^  and 

*;=ij  ■ 

Clearly 

“iO  " i ■ 1, . . . .n  . 

. At  the  beginning  of  the  jth  cycle  of  the  algorithm  the  following  data  is  available: 
* •*'  positive  constants  y y y,  6j  and  6^  with  * *2  * 

4^  < 0.5,  Y < 1.  Furthersxjre,  the  set  Jlx^)  and  the  matrix  are  given.  The  jth 

cycle  of  the  algorithm  consists  of  the  following  3 steps. 

Step  1 : Computation  of  the  search  direction  Sy  Compute  ^ with 

> 0 and  determine  k • such  that 


All  1 with  > 0 


•j  ■ ‘^kj  *">  ■ ^^1  ' 


otherwise  set 


I and  - y^  • 


If  - 0,  stop)  otherwise  qo  to  Step  2. 

Step  2;  Cosiputation  of  the  step  size If  i ® * " !*•••»■  ■ •, 

otherwise  set 


, ^ a|K  - (b)  . I 

3 • min/  — — ^1 

J V “I“j  I 


for  all  i with  a' 


!•)  ■ "} 


Determine  such  that 


and 


F(x^  - 5js^)  i.F(x^)  - 
(VF(x^  - ^^B^))*Sj  < i^q-s^ 


with  Oj  « 1 if  possible.  Set 


Oj  » min{o^,o^}  and  x^^j^  " *J  ~ °j*j 


Cooipute  Bnd  qo  to  Step  3. 

Step  3:  Computation  of 

r • 

Case  1:  s.  - 2.  (c!  .g  )c  and  a < o , (no  change  inthe  set  of  active  constraints) 

3 ^ i3  j ij  3 3 


For  all  i with  > 0 set 


For  all  i with  - 0 compute 


‘'i.j+1  “ "13  " 


“13  ■ Bj(q^  - g^^^) 


and  set 


Let 


nj’3^1  - 'll’ 


'i.j+l  ' 'ij  * "ij*!  ' 


'^j+l  “ *'l,j+l'""'n, j+l*'  ■’'“j+l*  " ' 


replace  j with  1 t 1 and  go  to  Step  1. 


— — “j  ■ ‘^kj  dropping  an  activ*  constraint).  Sst 


C * C 

i.J*l  'ij 

tor  i • k 

and  all 

i with  Oj^  - 0 . 

’ 'll 

. Phi’Ll  e 

for  all 

1 ^ k with  > 0 

‘'l.j+l'--- 

''n.J*l’ 

"‘Vi' 

■ *“l.j*l “n,J*l^ 

'"i.  j+1 

- for 

i - 1,-.. 

.n,  i # k 

“k.j+1 

- 0 . 

Replace  j with  j ♦ 1 and  go  to  Step  1. 

^ °j  " “j'  <*dding  a new  active  constraint).  Let 

“ij“° 

Sj  be  the  gradient  of  the  new  active  constraint.  Select  any  v with  a ^ - 0 and 
conpute 


^.0  “j  ■ ''j"t  • 


Vt’'  • 


If  • 0 set 


otherwise 


Coaputc 


and  set 


'"i.j+l  ■ ‘'ij  i “ith  - 0 , 


[1  - t.c*  a 1 

v>t  ' 

‘j  “ - ‘=;a]- 

“t  " *t  - ^ , 

rt  %n  *j  * '*44 
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Set 


where 


'’t'l 


■ 'ij  ■ *“  " “ij  * ° 


■ '^l.M ■"'’‘jn*  • S,j4l “n.J*!’ 


“l.j+l  ’ “ij-  ^ ‘ ‘ “ '' 

Vi^i  ■ ' • 

Replace  j with  j + 1 and  go  to  Step  1. 

* 

Case  4:  s , ■ c.  , and  a.  • o.,  (adding  and  dropping  an  active  constraint).  If 
3 kj  j j 

“ij  » 0-  i - 1 " 

k.j+1 


'i,j+l  '-ij  a-c^  '■ki' 


I,  i ^ k . 


If  at  least  one  » 0 use  the  procedure  given  in  Case  3 to  cooipute  a new  matrix  ^j+i* 

Then  use  the  method  of  Case  2 witli  C^  and  replaced  with  *•  respec- 
tively, to  determine  *-j^.i  ‘^**j+l*‘  j with  j ♦ 1 and  go  to  Step  1. 

Remark: 


i)  In  Step  1 we  set  y , ■ yy,  < y.  whenever  s • c i.e.,  whenever  an  active 
constraint  is  dropped.  Since  in  the  convergence  proof  we  only  use  the  fact  that 

is  a convergent  sequence  of  positive  nuisbers  with 

lim  y “ 0 iff  8.  " c . for  infinitely  many  j 
j^«  J J Rj 

any  method  which  produces  a sequence  with  these  properties  can  be  used. 

ii)  The  algorithm  can  easily  be  modified  to  handle  linear  equality  constraints. 
Since  equality  constraints  are  always  active  the  only  difference  is  that  a vector  c^^ 
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corraspondinq  to  an  equality  constraint  is  not  a candidate  for  the  search  direction  s 
in  Step  1 of  the  alqorithsi.  This  could  bo  indicated  by  choosinq  • -1  for  all  i 
such  that  corresponds  to  an  equality  constraint. 
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4.  Super  linear  convergence 

First  we  observe  that  for  each  and  *j+j  generated  by  the  algor ithn  we  have 

x^,  *1*1  * *^**j*l*  * F(Xj).  Furthermore,  the  algorithm  terminates  with  an  x^ 

if  and  only  if  x^  satisfies  the  Kuhn-Tucker  conditions  and  is  therefore  an  optimal 
solution. 

We  assume  now  that  the  algorithm  generates  an  infinite  sequence  (x^)  and  shall 
prove  that,  under  the  assumption  stated  in  Section  2,  this  sequence  converges  super- 
linearly  to  the  optimal  solution  z.  The  convergence  proof  is  closely  related  to  the 
proof  given  by  Fischer  14). 

2 ! There  is  and  I C {l,...,m)  such  that,  for  j ^ 


(b) 


i’ 


i e I 


ajx^  < (b)^.  i / I . 

Proof ; For  every  j let  C (l,...,m)  be  such  that  i r if  and  only  if 

a'x.  ■ (b)..  Furthermore,  let  J C {0,1,..,}  be  such  that  an  active  constraint  is 
ID  1 

dropped  at  x^  if  and  only  if  j t J. 

Suppose  that  the  lemma  is  not  true.  Then  J is  an  infinite  set  and 
n • {l  C {l,...,m}ll  “ for  infinitely  many  J r j) 

is  non-empty.  Choose  any  i * fl  which  has  the  maximal  number  of  elements  of  all  I C n. 
Set 

Jj  - (j  * J|l^  - i)  . 

There  is  k C i and  an  infinite  subset  C such  that  for  each  x^,  j * J^, 
always  the  constraint  aj^x  £ (b)|^  is  dropped  from  the  set  of  active  constraints.  Since 
Yj  -►  0 as  j -►  » it  follows  from  Step  1 of  the  algorithm  that  g^  0 as  J ■*  *, 
j € J^,  where  g^  denotes  the  orthogonal  projeotion  of  g^  onto 

T - {x|a'x  - 0,  i e I)  . 

By  the  uniform  convexity  of  F(x)  this  implies 


where  x f R is  the  unique  solution  of 


min(F(x) la^x  - (b)^,  I » l)  , 


^^(x)  “ X a and  X ■ max{X.,  i * I)  . 
i<I  ^ 

First  suppose  that  X ^ 0.  By  Taylor's  theorem  there  are  numbers  0 t ^ 

such  that  for,  j < J^, 

* 'kj®'*  * S‘’'j  ■ 

1 ll-\jll  II  f'(x  + - *))|lll  Xj  - ill  1 nllcj^jll  II  Xj  - ill 


- i)'g^  - (Xj  - i)'7F(ii)  ♦ (x^  - ii)G(;  + n^(x^  - ;))(Xj  - i)  >u||x^-  i||^. 
Thus  (x^  - x)'g^  - (Xj  - i) 'g^  implies  ||g^||  > li||x^-  x||,  and  we  have 

"’l"  "2  • 

Since  -►  0 as  j • we  obtain  the  contradiction  that  for  i ' Jj 
large  no  active  constraint  is  dropped. 

To  complete  the  proof  it  suffices,  therefore,  to  show  that  X ^ 0.  Suppose  X > 0. 


'"kj’j  “ ^ ^ j 1 * -^2 

and  Sj  - C|^^  for  j < J^,  it  follows  that 

gJXj  i *ll*jll  f<Jr  J * ^2  some  f > 0 . 

Since  F(x)  is  bounded  from  below  step  2 of  the  algorithm  implies  that 

0*  -*  0 as  J •*  -,  J * 


* 

for  infinitely  many  j * Jj  • 

Thus  there  is  t 1 {l,...,m)  ■ 1 and  an  infinite  subset  C such  that  the  constraint 

a'x  (b)^  is  active  at  x^^j,  j « Jj.  Furthersore  aji  - (b)^  since 
llXj^l  - XjK  ♦ 0 as  j J » Jj. 
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For  j « J lat  T 2.  0 be  the  largest  integer  such  that  for  s x 

33  ) 3 J 

the  sane  constraints  are  active.  Since  the  definition  of  I in|.l  ies  that  foi  all  Out 

at  oi''St  finitely  isany  **311  active  constraint  is  dro^^a.sl  it  follcars  again  fron 

Step  1 of  the  algorithm  that 


j+r 


as  j » 3 ‘ J, 


J 


vrhere  x is  the  unique  solution  of 

(4.2)  inln{F(x)  |a|x  • (b)^t  i « I - (li)  ♦ (1))  . 

I.et  I*  - I - (k)  + (t}  and 

(4.3)  7F(x*)  - I t a . 

• * ^ 

it  I 

Because  x is  a feasible  solution  of  problem  (4.2)  and  F(x)  and  F(x  ) are 
both  cluster  points  of  the  monouine  decreasing  sequence  (F(xJ)  it  follows  that 


F(x)  • F(x  ) and  x 


. Subtracting  (4.3)  from  (4.2)  we  have  therefore 


i.e,.  ■ 3, a.  ■ 0 < 
)c  k t t 


which  by  the  linear  independence  of  gradients  of  active  constraints  gives  the  contradic- 


tion that  \ 


Theorem:  Let  Assumption  1 be  satisfied.  The  sequence  (x^)  converges  superlinearly 
to  the  optimal  solution  of  the  problem 

mi  n(F(x)  I Ax  ^ b)  . 

For  j sufficiently  large  0^  • 1. 

Proof:  Let  and  I be  defined  as  in  Lemma  2.  For  3 1 Iq  the  application  of  the 

algorithm  to  tlie  given  problem  is  equivalent  to  its  application  to  the  problem 

(4.4)  min(F(x)|a^x  - (b)^,  1 f I)  . 

Therefore/  it  follows  from  Theorem  2.1  in  14)  that  « 1 for  3 sufficiently  large 

and  that  (x^)  converges  superlinearly  to  tlM  optisial  solution  t of  (4.4).  Lat 

VF(z)  - I 3. a , 
ifl  ^ 
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I • 


I 


and  let  9^  denota  the  orthoqonal  projection  of  9^  onto  {x|a'x  • 0,  i « Z).  Sine* 


for  aonc  y >0,  llq^H  ♦0  j ■» 


and 


“A  - BaafA^,  i * I)  as  J -► 


It  follows  froB 


l9jll  > ^ - ^0'  A < 0.  Sines  a ( R, 

optisial  solution  of  the  given  problea. 


it  is  the 


I 
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